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PART  I 

a.  Number  of  papers  submitted  to  refereed  journals,  but  not  published:  2 


b.  +Number  of  papers  published  in  refereed  journals:  12 

Graesser,  A.C.,  &  Bertus.  E.L.  (1998).  The  construction  of  causal  inferences  while  reading  expository 
texts  on  science  and  technology.  Scientific  Studies  of  Reading.  2.  247-269,, 

Graesser.  A.C.,  Kassler,  M.A.,  Kreuz.  R.J.,  &  McLain-Alien,  B.  (1998).  Verification  of  statements  about 
story'  worlds  that  deviate  from  normal  conceptions  of  time:  What  is  true  about  Einstein's  Dreams ? 
Cognitive  Psychology,  35,  246-301. 

Wiemer-Hastings,  K.  &  Graesser,  A.C.  (1998).  Who  needs  created  features?  Brain  and  Behavior 
Sciences.  21.  39. 

Williams,  K.E.,  Hultman.  E.,  &  Graesser,  A.C.  (1998).  CAT:  A  tool  for  eliciting  knowledge  on  how  to 
perform  procedures.  Behavior  Research  Methods.  Instruments.  &  Computers.  30,565-572. 

Zwaan.  R.  A.,  &  Graesser,  A.C.  (1998).  Constructing  meaning  during  reading.  Scientific  Studies  of 
Reading.  2,  195-198. 

Graesser,  A.C.,  Bowers,  C.,  Olde,  B.,  White,  K.,  &  Person,  N.  (1999).  Who  knows  what:  Propagation  of 
knowledge  among  agents  in  a  literary  storyworld.  Poetics:  Journal  of  Empirical  Research  on 
Literature,  the  Media,  and  the  Arts.  26.  143-175. 

Krovi,  R.,  Graesser,  A.C.,  and  Pracht,  W.E.  (1999).  Agent  behaviors  in  virtual  negotiation 
environments.  IEEE  Transactions  on  Systems.  Man,  and  Cybernetics.  29.  15-25. 
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Franklin,  SB..  &  Graesser.  A.C.  (in  press).  A  software  agent  model  of  consciousness.  Consciousness  and 
Cognition. 

Graesser.  A.C..  Bowers.  C.A..  Olde.  B..  &  Pomeroy,  V.  (in  press).  Who  said  what?  Source  memory  for 
narrator  and  character  agents  in  literary  short  stories.  Journal  of  Educational  Psychology. 

Magliano.  J.,  Trabasso,  T.,  &  Graesser,  A.C.  (in  press).  Strategic  processing  during  comprehension. 
Journal  of  Educational  Psychology. 

Ottati.  V..  Rhoads,  S.,  &  Graesser.  A.C.  (in  press).  The  effect  of  metaphor  on  processing  sty  le  in  a 
persuasion  task:  A  motivational  resonance  model.  Journal  of  Personality  and  Social  Psychology. 

Wiemer-Hastings.  K.,  &  Graesser.  A.C.  (in  press).  Perceiving  abstract  concepts.  Brain  and  Behavior 
Sciences. 


c.  +Number  of  books  or  chapters  submitted,  but  not  yet  published:  4 


d.  +Number  of  books  or  chapters  published:  20 
Books: 

Hacker,  D.J.,  Dunlosky.  J„  &  Graesser,  A.C.  (1998)(Eds.).  Metacognition  in  educational  theory  and 
practice.  Mahwah.  NJ:  Erlbaum. 

Goldman.  S..  Graesser,  A.C.,  &  van  den  Broek,  P.  (1999)(Eds).  Narrative  comprehension,  causality,  and 
coherence.  Mahwah,  NJ:  Erlbaum. 

Chapters: 

Graesser.  A.C.,  &  Tipping,  P.  (1998).  Understanding  texts.  In  W.  Bechtel  and  G.  Graham  (Eds.),  A 
companion  to  cognitive  science,  (pp.  324-330).  Oxford:  Blackwell. 

Graesser,  A.C.,  Bowers,  C.A.,  Hacker,  D.J.,  &  Person,  N.  K.  (1998).  An  anatomy  of  naturalistic  tutoring. 
In  K.  Hogan  and  M.  Pressley  (Eds.),  Scaffolding  of  instruction.  Brookline  Books. 

Graesser,  A.C.,  Kennedy,  T„  Wiemer-Hastings.  P.,  &  Ottati,  V.  (1999).  The  use  of  computational 
cognitive  models  to  improve  questions  on  surveys  and  questionnaires.  In  M.G.  Sirken,  D.J. 
Hermann.  S.  Schechter,  N.  Schwarz,  J.M.  Tanur,  &  R.  Tourangeau  (Eds.),  Cognition  and  survey 
methods  research  (pp  69-86).  New  York:  Wiley. 

Graesser,  A.C.,  Kennedy,  T.„  Wiemer-Hastings,  P„  &  Ottati,  V.C.  (1999).  The  use  of  computational 
cognitive  models  to  improve  questions  on  surveys  and  questionnaires.  In  M.G.  Sirken,  T.  Jabine,  G. 
Willis,  E.  Martin,  &  C.  Tucker  (Eds),  A  new  agenda  for  interdisciplinary  survey  research  methods 
(p.  28).  Hyatstsville,  MD:  US  Department  of  Health  and  Human  Services. 
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Person.  N.K.  &  Graesser.  A.C.  (1999).  Evolution  of  discourse  in  cross-age  tutoring  In  A.M.  O'Donnell 
and  A.  King  (Eds.),  Cognitive  perspectives  on  peer  learning  (pp.69-86).  Malnvah.  NJ:  Erlbauin. 

Graesser.  A.C.,  Bovvers.  C.  A..  Bayen,  U.J..  Hu.  X.  (in  press).  Who  said  what?  Who  knows  what? 
Tracking  speakers  and  knowledge  in  narrative.  In  W.  van  Peer  &  S.  Chatman,  (Eds.).  Narrative 
perspective:  Cognition  and  emotion.  New  York:  SUNY  Press. 

Graesser.  A.C.,  Pomeroy,  V.,  &  Craig.  S.  (in  press).  Psychological  and  computational  research  on  theme 
comprehension.  In  W.  van  Peer  and  M.M.  Louwerse  (Eds.).  Thematics  in  psychology'  and  literary' 
studies.  Albany:  SUNY  Press. 

Graesser.  A.C..  &  Wieiner-Hastings.  K.  (1999).  Situation  models  and  concepts  in  story  comprehension. 

In  Goldman.  S.R..  Graesser.  A.C.,  and  van  den  Broek,  P.  (Eds).  Narrative  comprehension,  causality. 
and  coherence  (pp.  77-92).  Malnvah.  NJ:  Erlbaum. 

Graesser.  A.C..  Klettke,  B.,  &  Olde.  B.A.  (in  press).  How  does  the  mind  construct  and  represent  stories? 
In  M.  Green.  J.  Strange,  and  T.  Brock  (Eds.),  Narrative  impact:  Social  and  cognitive  foundations. 
Malnvah:  NJ:  Erlbaum. 

Graesser.  A.C..  Wiemer-Hastings.  P..  &  Wiemer-Hastings.  K.  (in  press).  Constructing  inferences  and 
relations  during  text  comprehension.  In  T.Sanders,  J.  Schilperoord,  &  W.  Spooren  (Eds.),  Text 
representation:  Linguistic  and  psycholinguists  aspects.  Amsterdam:  Benjamins. 

Magliano.  J.P.,  Zwaan,  R.A.,  &  Graesser,  A.C.  (1999).  The  role  of  situational  continuity  in  narrative 
understanding.  In  S.R.  Goldman  and  H.  van  Oostendorp  (Eds.),  The  construction  of  mental 
representations  during  reading.  Malnvah,  NJ:  Erlbaum. 

Published  Conference  Proceedings: 

Graesser.  A.C..  Franklin.  S..  &  Wiemer-Hastings.  P.  and  the  Tutoring  Research  Group  (1998). 

Simulating  smooth  tutorial  dialogue  with  pedagogical  value.  Proceedings  of  the  American 
Association  for  Artificial  Intelligence  (pp.  163-167).  Menlo  Park,  CA:  AAAI  Press. 

Hu.  X..  Graesser.  A.C..  and  the  Tutoring  Research  Group  (1998).  Using  WordNet  and  latent  semantic 
analysis  to  evaluate  the  conversational  contributions  of  learners  in  tutorial  dialog.  Proceedings  of  the 
International  Conference  on  Computers  in  Education.  Vol.  2  (pp.  337-341).  Beijing,  China:  Springer. 

McCauley,  L„  Gholson,  B„  Hu,  X..  Graesser.  A.C..  and  the  Tutoring  Research  Group  (1998).  Delivering 
smooth  tutorial  dialogue  using  a  talking  head.  Proceedings  of  the  Workshop  on  Embodied 
Conversation  Characters  (pp.  31-38).  Tahoe  City,  CA:  AAAI  and  ACM. 

Wiemer-Hastings,  K.,  &  Graesser,  A.C.  (1998).  Contextual  representations  of  abstract  nouns:  A  neural 
network  approach.  Proceedings  of  the  20th  Annual  Conference  of  the  Cognitive  Science  Society  (pp. 
1036-1041).  Malnvah,  NJ:  Erlbaum. 

Wiemer-Hastings,  P„  Graesser,  A.C.,  &  Wiemer-Hastings.  K.  (1998).  Inferring  the  meaning  of  verbs 
from  context.  Proceedings  of  the  20th  Annual  Conference  of  the  Cognitive  Science  Society  (pp. 
1142-1147).  Mahwah,  NJ:  Erlbaum. 
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Wieiner-Hastings.  P..  Graesser.  A.C.,  Harter,  D..  and  the  Tutoring  Research  Group  (1998).  The 
foundations  and  architecture  of  AutoTutor.  Proceedings  of  the  4th  International  Conference  on 
Intelligent  Tutoring  Systems  (pp.  334-3411  Berlin.  Germany  Springf>r-V^rlag 

Zhang.  Z..  Franklin.  S„  Olde.  B..  Wan,  Y..  &  Graesser,  A.C.  (1998).  Natural  language  sensing  for 
autonomous  agents.  IEEE  Proceedings  on  Intelligence  and  Systems. 

Corbett.  A..  Anderson.  J.,  Graesser.  A..  Koedinger.  K..  &  van  Lehn,  K.  (1999).  Third  generation 
computer  tutors:  Learn  from  or  ignore  human  tutors?  Proceedings  of  the  1999  Conference  of 
Computer-Human  Interaction  (pp.  85-86).  New  York:  ACM  Press. 

Olde.  B.  A..  Hoeffner.  J„  Chipman.  P„  Graesser.  A.C.,  and  the  Tutoring  Research  Group  (1999).  A 
connectionist  model  for  part  of  speech  tagging.  Proceedings  of  the  American  Association  for 
Artificial  Intelligence  (pp.  172-176).  Menlo  Park,  C A:  AAAI  Press. 

Wieiner-Hastings,  P..  Wiemer-Hastings,  K„  Graesser.  A.,  and  the  Tutoring  Research  Group  (1999). 
Approximate  natural  language  understanding  for  an  intelligent  tutor.  Proceedings  of  the  American 
Association  for  Artificial  Intelligence.  Menlo  Park,  CA:  AAAI  Press. 


e.  +Number  of  printed  technical  reports/non-referred  papers:  2 

Graesser.  A.C.,  &  Hoeffner,  J,  (in  press).  Complex  computer  models  are  needed  to  understand  complex 
learning.  Contemporary  Psychology. 

Graesser.  A.C..  &  Whitten,  S.  (in  press).  Review  of  Walter  Kintsch’s  book,  “Comprehension:  A 
paradigm  in  cognition.”  Journal  of  Pragmatics. 


f.  +Number  of  patents  filed:  0 


g.  +Number  of  patents  granted:  0 


h,  +Number  of  invited  presentations:  8 

Graesser.  A.C.  (July,  1998).  Eight  cool  things  from  the  collocated  conferences.  Invited  panel  at  the 
Fifteenth  National  Conference  on  Artificial  Intelligence,  Madison,  WI. 

Graesser,  A.C.,  &  Kreuz,  R.  (1998,  July).  Discourse  in  contexts  of  learning.  Invited  tutorial  at  the  20th 
Annual  Conference  of  the  Cognitive  Science  Society,  Madison,  Wisconsin. 

Graesser,  A.C.,  Craig,  S.,  &  Pomeroy,  V.  (1998,  August).  How  to  peddle  literature  to  the  hardnose 
experimental  psychologist.  Invited  presentation  at  the  International  Association  for  the  Empirical 
Studies  of  Literature,  Art,  and  Media,  Utrecht,  the  Netherlands. 

Graesser.  A.C.  (1999,  February).  Deep  comprehension  of  illustrated  texts  about  everyday  devices. 
Colloquium  at  the  University  of  Coimbra,  Coimbra,  Portugal. 
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Graesser.  A  C.  (1999,  February).  Question-driven  explanatory  reasoning  about  devices  that  malfunction. 
Progress  report  to  the  Navy  on  Office  of  Naval  Research  grant.  Oxford,  Mississippi. 

Graesser.  A  C.  and  Hacker.  D.  (1999.  April).  AutoTutor:  An  automated  computer  tutor.  Invited 

symposium  on  intelligent  tutoring  systems  at  the  meetings  of  the  American  Educational  Research 
Association.  Montreal.  Canada. 

Graesser.  A.C..  Craig.  S..  Pomeroy.  V..  &  Olde.  B.  (1999,  April).  Deep  comprehension  of  illustrated  texts 
in  the  context  of  a  breakdown  scenario.  Invited  symposium  on  discourse  comprehension  at  the 
meetings  of  the  American  Educational  Research  Association.  Montreal.  Canada. 

Graesser.  A.C.  (1999.  May).  Invited  panel  symposium  on  intelligent  tutoring  systems.  Symposium  at  the 
Computer  Human  Interaction  Conference,  Pittsburgh,  PA. 


i.  +Number  of  submitted  presentations:  12 

Graesser,  A.C.,  Franklin.  S„  and  the  Tutoring  Research  Group  (1998,  July).  The  goals  and  design  of 
AutoTutor.  Symposium  presented  at  the  8th  Annual  Meeting  of  the  Society  for  Text  and  Discourse. 
Madison.  Wisconsin. 

Wiemer-Hastings.  K.,  &  Graesser.  A.C.  (1998.  July).  Contextual  representations  of  abstract  nouns:  A 
neural  network  approach.  Poster  presented  at  the  20th  Annual  Conference  of  the  Cognitive  Science 
Society.  Madison.  Wisconsin. 

Wiemer-Hastings.  P..  Graesser,  A.C.,  &  Wiemer-Hastings,  K.  (1998,  July).  Inferring  the  meaning  of  verbs 
from  context.  Poster  presented  at  the  meetings  of  the  20th  Annual  Conference  of  the  Cognitive 
Science  Society,  Madison,  Wisconsin. 

Wiemer-Hastings,  P.,  Graesser,  A.C.,  Harter,  D„  and  the  Tutoring  Research  Group  (1998,  August).  The 
foundations  and  architecture  of  AutoTutor.  Paper  presented  at  the  4th  International  Conference  on 
Intelligent  Tutoring  Systems,  An  Antonio,  Texas. 

Hu,  X..  Graesser,  A.C..  and  the  Tutoring  Research  Group  (1998,  October).  Using  WordNet  and  latent 
semantic  analysis  to  evaluate  the  conversational  contributions  of  learners  in  tutorial  dialog.  Paper 
presented  at  the  International  Conference  on  Computers  in  Education,  Beijing,  China. 

McCauley,  L„  Gholson,  B„  Hu,  X.,  Graesser.  A.C.,  and  the  Tutoring  Research  Group  (1998,  October). 
Delivering  smooth  tutorial  dialogue  using  a  talking  head.  Paper  presented  at  the  Workshop  on 
Embodied  Conversation  Characters  (AAAI  and  ACM).  Tahoe  City,  CA. 

Wiemer-Hastings.  K„  Wiemer-Hastings,  P„  &  Graesser,  A.  C.  (1998,  November).  The  more,  the  better? 
The  effect  of  the  number  of  input  variables  on  generalization  in  neural  networks.  Paper  presented  at 
the  28th  Annual  Conference  of  the  Society  for  Computers  in  Psychology,  Dallas,  Texas. 

Wiemer-Hastings,  P„  Wiemer-Hastings,  K.,  &  Graesser,  A.C.  (1999,  January).  Using  latent  semantic 
analysis  to  evaluate  tutee  contributions.  Paper  presented  at  the  Tenth  Annual  Meeting  of  the  Winter 
Conference  on  Discourse,  Text  &  Cognition,  Jackson  Hole,  Wyoming. 
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Graesser.  A.C..  Pomeroy,  V.,  &  Craig,  S.  (1999,  January).  Think  aloud  protocols  in  the  context  of 

illustrated  texts  and  devices  that  malfunction.  Paper  presented  at  the  Tenth  Annual  Meeting  of  the 
Winter  Conference  on  Discourse.  Text  &  Cognition.  Jackson  Hole,  Wyoming. 

Olde.  B  A..  Hocffner.  J..  Chipman.  P„  Graesser.  A.C..  and  the  Tutoring  Research  Group  (1999.  May).  A 
connectionist  model  for  part  of  speech  tagging.  Paper  presented  at  the  12th  International  Florida 
Artificial  Intelligence  Research  Conference.  Orlando.  Florida. 

Wiemer-Hastings.  P„  Wiemer-Hastings.  K..  and  Graesser,  A.  (1999,  May).  Approximate  natural 

language  understanding  for  an  intelligent  tutor.  Paper  presented  at  the  12th  International  Florida 
Artificial  Intelligence  Research  Conference.  Orlando,  Florida.  * 

Corbett.  A.,  Anderson.  J„  Graesser.  A..  Koedinger.  K„  &  van  Lehn.  K.  (1999,  May).  Third  generation 
computer  tutors:  Learn  front  or  ignore  human  tutors?  Panel  symposium  at  the  1999  Computer- 
Human  Interaction  Conference.  Pittsburgh.  PA. 


j.  +Honors/Awards/Prizes  for  contract/grant  employees:  3 

Board  of  Visitors  Eminent  Faculty  Award,  University  of  Memphis.  May.  1999. 

Editor  of  the  journal  Discourse  Processes.  1997-2004. 

1999  Program  Chair  for  Division  C  (Learning  and  Instruction)  of  the  American  Educational  Research 
Association  Conference,  Montreal  Canada,  April  1999. 


k.  -t-Total  number  of  Full-time  equivalent  Graduate  Students  and  Post-Doctoral 
associates  supported  during  this  period,  under  the  PR  number: 

Graduate  students:  2  (1  female,  1  male,  0  Asian,  0  Minority) 
Post-doctoral  Associates:  0 


I.  Other  funding 

All  of  the  following  projects  investigate  question  asking  and  question  answering  mechanisms,  which  is 
the  direct  focus  of  the  ONR  grant. 

National  Science  Foundation,  Simulating  tutors  with  natural  dialog  and  pedagogical  strategies.  1997- 
2000.  $900,000  (Dr.  Graesser  is  PI,  $210,000  received  this  year).  This  project  involves  developing  an 
automated  computer  tutor  (called  AutoTutor)  that  asks  and  answers  questions;  question  asking  and 
answering  is  the  direct  focus  of  the  ONR  grant,  so  the  projects  are  quite  related.  Also,  both  projects 
involve  the  technical  domains;  the  NSF  grant  tutors  students  on  computer  literacy  whereas  the  ONR  grant 
investigates  mechanical  and  electronic  devices. 


United  States  Bureau  of  Census.  QUEST  questionnaire  evaluation  tool  1998-1999.  $58,512  (Dr.  Gracsser 
is  PI:  $30,000  received  this  year).  This  project  involves  building  a  computer  tool  that  critiques  questions 
on  surveys  and  questionnaires.  Question  asking  and  answering  is  the  direct  focus  of  the  ONR  grant  so 
these  projects  are  quite  related. 

National  Institutes  of  Health.  Smoking  cessation  in  the  elderly.  1998-2001.  $1,600,000  (Dr.  Bob  Klesges 
is  PI).  As  a  co-Pl.  my  role  is  to  recommend  changes  on  the  question  asking,  question  answering,  and 
discourse  interaction  styles  of  health  educators  who  try  to  get  elderly  smokers  to  stop  smoking.  Question 
asking  and  answering  mechanisms  are  investigated  in  both  the  ONR  and  NIH  projects. 

Office  of  Naval  Research.  Intelligent  distributed  agents.  1998-2002.  $1,500,000  (Dr.  Stan  Franklin  is  PI). 
As  a  co-PI,  my  role  is  to  advise  graduate  students  how  to  design  question  asking  and  question  answering 
interfaces  on  software  agents,  and  how  to  evaluate  the  system  performance. 
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PART  U 


a.  Principal  investigator:  Arthur  C.  Graesser 

b.  Telephone  Number:  (901)  678-2742 

c.  ONR  Program  Officer:  Jan  Dickieson 

d.  Program  objective: 

The  goal  of  this  project  is  to  investigate  how  adults  reason  about  everyday  devices  (e  g., 
locks,  doorbells,  dishwashers,  clutches)  when  the  devices  break  down.  Deep 
comprehension  of  a  device  is  presumably  manifested  by  reasoning  in  the  face  of  the 
malfunctioning  device,  because  the  comprehender  has  to  diagnose  the  fault  and  figure  out 
how  to  correct  it.  The  primary  hypothesis  is  that  good  comprehenders  will  ask  and 
answer  good  questions.  A  good  question  is  defined  as  a  question  that  identifies  a  plausible 
fault  that  causally  explains  the  breakdown.  A  bad  question  addresses  irrelevant  content 
that  does  not  provide  a  likely  explanation  of  the  malfunction. 

We  collected  data  from  108  college  students  at  the  University  of  Memphis.  The 
participants  read  6  illustrated  texts  on  everyday  devices:  a  cylinder  lock,  an  electronic  bell, 
a  car  temperature  gauge,  a  clutch,  a  toaster,  and  a  dishwasher.  The  device  mechanisms 
were  extracted  from  a  book  with  illustrated  texts,  The  Way  Things  Work  (Macaulay, 
1988).  After  reading  about  each  device,  the  participants  subsequently  received  scenarios 
in  which  the  device  breaks  down  (e.g.,  in  the  context  of  a  cylinder  lock,  suppose  that  the 
key  turns,  but  the  bolt  does  not  move).  During  this  time,  they  are  asked  either  to  “think 
aloud”  or  to  generate  questions  for  two  minutes  while  attempting  to  diagnose  and  repair 
the  malfunctions.  We  examined  the  questions  that  learners  ask  and  answer  when  they  are 
given  the  breakdown  scenario.  We  measured  the  volume  and  quality  of  the  content 
produced  during  the  question  asking  or  think  aloud  task.  The  content  of  the  questions  was 
also  mapped  onto  conceptual  graph  structures  (e.g.,  semantic  networks,  knowledge 
structures)  for  the  illustrated  texts  on  devices;  these  structures  include  component 
hierarchies,  spatial  region  hierarchies,  causal  chains/networks,  goal/plan/action  hierarchies, 
and  property  descriptions  that  are  depicted  in  either  text  or  picture  form 

After  providing  the  question  asking  and  think  aloud  protocols  for  all  6  devices,  the 
participants  completed  an  objective  test  on  their  understanding  of  the  devices.  This 
consisted  of  six  3 -alternative,  forced-choice  questions  about  each  device  (36  total 
questions).  Following  this  objective  test  of  device  comprehension,  participants  completed 
a  battery  of  tests  that  measured  their  cognitive  abilities  and  personality.  The  tests  of 
cognitive  ability  include  the  ASVAB  (the  Armed  Services  Vocational  Aptitude  Battery), 
working  memory  span,  spatial  reasoning,  and  exposure  to  print.  Male  and  female 
participants  also  completed  the  NEO  inventory,  which  measures  individuals  on  the  “big 
five”  personality  factors:  neuroticism,  extroversion,  openness,  agreeableness,  and 
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conscientiousness.  We  examined  whether  these  measures  of  individual  differences  could 
predict  comprehension  scores  and  measures  of  question  asking. 

e.  Significant  Results  during  the  Last  Year 

The  results  of  the  study  supported  the  hypothesis  that  deep  comprehenders  of  devices  ask 
good  questions  when  confronted  a  breakdown  scenario.  Mechanical  comprehension  and 
gender  were  the  two  most  robust  predictors  of  both  the  device  comprehension  scores 
(predicting  44%  of  the  variance)  and  the  quality  of  questions  in  the  question  asking  task 
(predicting  38%  of  the  variance)  when  multiple  regression  analyses  were  conducted.  The 
profile  of  correlations  with  question  quality  were  nearly  identical  with  the  profile  of 
correlations  with  device  comprehension  scores.  Question  quality  is  therefore  an  excellent 
litmus  test  of  how  well  a  college  student  understands  the  mechanisms  underlying  a  device. 
The  volume  of  content  in  the  question  asking  task  and  the  think  aloud  task  is  a  poor  index 
of  deep  comprehension.  Similarly,  the  quality  of  content  in  the  think  aloud  task  is  also  not 
a  good  index  of  deep  comprehension.  The  questions  asked  by  students  with  high 
mechanical  comprehension  scores  had  two  characteristics,  (a)  the  questions  converged  on 
components  in  the  mechanism  that  are  plausible  faults  and  (b)  the  questions  had  a  more 
fine-grained  elaboration  of  the  parts,  processes,  and  relations  that  specified  how  the 
breakdown  occurred.  Stated  differently,  there  was  high  convergence  and  high  mechanistic 
detail. 

f.  Summary  of  Plans  for  Next  Year’s  Work 

Experiments  will  be  conducted  that  collect  eye  tracking  data  from  college  students  while 
they  read  illustrated  texts  about  everyday  devices,  and  while  they  generate  questions  when 
these  devices  break  down.  Participants  with  high  mechanical  comprehension  are  predicted 
to  have  their  eye  fixations  concentrate  on  faulty  components  and  processes  whereas  those 
with  low  mechanical  reasoning  should  have  more  undiscriminating  patterns  of  eye 
fixations.  More  generally,  we  will  relate  the  patterns  of  eye  tracking  to  elements  in  the 
illustrated  texts,  to  nodes  in  the  conceptual  graph  structures,  and  to  measures  of  individual 
differences.  We  will  investigate  the  cognitive  mechanisms  that  explain  question  asking  and 
eye  fixations  in  the  context  of  a  breakdown  scenario.  There  is  a  more  practical 
implication:  A  quick  way  to  find  out  whether  a  sailor  has  a  talent  for  understanding  or 
operating  a  device  is  to  present  a  break  down  scenario  and  to  record  the  sailor’s  question 
asking  behavior  and  eye  fixations. 

g.  Names  of  Graduate  Students  Currently  Working  on  the  Project: 

Victoria  Pomeroy 
Brent  Olde 

Shulan  Lu  (starting  July,  1999) 
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PART  III 


These  are  the  three  view  graphs:  An  introductory  5-part  viewgraph  and  two  supporting 
viewgraphs.  The  paragraph  descriptions  are  below.  A  Powerpoint  file  with  the  graphs  is 
in  a  separate  file. 

VIEWGRAPH  1 

The  goal  of  this  project  is  to  investigate  how  adults  reason  about  everyday  devices  (e.g., 
locks,  doorbells,  dishwashers,  clutches)  when  the  devices  break  down.  Deep 
comprehension  of  a  device  is  presumably  manifested  by  reasoning  in  the  face  of  the 
malfunctioning  device,  because  the  comprehender  has  to  diagnose  the  fault  and  figure  out 
how  to  correct  it.  The  primary  hypothesis  is  that  good  comprehenders  will  ask  and 
answer  good  questions.  A  good  question  is  defined  as  a  question  that  identifies  a  plausible 
fault  that  causally  explains  the  breakdown.  A  bad  question  addresses  irrelevant  content 
that  does  not  provide  a  likely  explanation  of  the  malfunction.  We  collected  data  from  108 
college  students  at  the  University  of  Memphis.  The  participants  read  6  illustrated  texts  on 
everyday  devices:  a  cylinder  lock,  an  electronic  bell,  a  car  temperature  gauge,  a  clutch,  a 
toaster,  and  a  dishwasher.  The  device  mechanisms  were  extracted  from  a  book  with 
illustrated  texts.  The  Way  Things  Work  (Macaulay,  1988).  After  reading  about  each 
device,  the  participants  subsequently  received  scenarios  in  which  the  device  breaks  down 
(e  g.,  in  the  context  of  a  cylinder  lock,  suppose  that  the  key  turns,  but  the  bolt  does  not 
move).  During  this  time,  they  are  asked  either  to  “think  aloud”  or  to  generate  questions 
for  two  minutes  while  attempting  to  diagnose  and  repair  the  malfunctions.  We  examined 
the  questions  that  learners  ask  and  answer  when  they  are  given  the  breakdown  scenario. 
We  measured  the  volume  and  quality  of  the  content  produced  during  the  question  asking 
or  think  aloud  task.  These  were  correlated  with  scores  on  a  device  comprehension  test 
and  20  measures  of  individual  differences. 

VIEWGRAPH  2 

The  results  of  the  study  supported  the  hypothesis  that  deep  comprehenders  of  devices  ask 
good  questions  when  confronted  a  breakdown  scenario.  Mechanical  comprehension  and 
gender  were  the  two  most  robust  predictors  of  both  the  device  comprehension  scores 
(predicting  44%  of  the  variance)  and  the  quality  of  questions  in  the  question  asking  (QA) 
task  (predicting  38%  of  the  variance)  when  multiple  regression  analyses  were  conducted. 
The  profile  of  correlations  with  question  quality  were  nearly  identical  with  the  profile  of 
correlations  with  device  comprehension  scores.  Question  quality  is  therefore  an  excellent 
litmus  test  of  how  well  a  college  student  understands  the  mechanisms  underlying  a  device. 
The  volume  of  content  in  the  question  asking  task  and  the  think  aloud  (TA)  task  was  a 
poor  index  of  deep  comprehension.  Similarly,  the  quality  of  content  in  the  think  aloud 
task  is  also  not  a  good  index  of  deep  comprehension. 
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VTEWGRAPH  3 


The  content  of  the  questions  was  also  mapped  onto  conceptual  graph  structures  (eg., 
semantic  networks,  knowledge  structures)  for  the  illustrated  texts  on  devices;  these 
structures  include  component  hierarchies,  spatial  region  hierarchies,  causal 
chains/networks,  goal/plan/action  hierarchies,  and  property  descriptions  that  are  depicted 
in  either  text  (T)  or  picture  form  (P).  The  questions  asked  by  students  with  high 
mechanical  comprehension  scores  had  two  characteristics;  (a)  the  questions  converged  on 
components  in  the  mechanism  that  are  plausible  faults  and  (b)  the  questions  had  a  more 
fine-grained  elaboration  of  the  parts,  processes,  and  relations  that  specified  how  the 
breakdown  occurred.  Stated  differently,  there  was  high  convergence  and  high  mechanistic 
detail  In  the  Powerpoint  figure,  the  nodes  with  red  dots  are  the  likely  faults  for  the 
breakdown  scenario  (The  key  turns  but  the  bolt  does  not  move). 
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Question-driven  Explanatory  Reasoning  About 
Devices  that  Malfunction 


OBJECTIVE 

To  assess  whether  an  adult  has  a  deep  comprehension  of  a 
device. 

To  explore  the  cognitive  mechanisms  that  underlie  deep 
comprehension  of  illustrated  texts. 

To  develop  models  of  human  question  asking  and  answering 

APPROACH 

Give  breakdown  scenarios  after  college  students  read 
illustrated  texts  from  The  Way  Things  Work. 

Collect  question  asking  and  think  aloud  protocols  in  the 
context  of  a  breakdown  scenario. 

Collect  measures  of  cognitive  ability  and  personality. 

ACCOMPLISHMENTS 

Participants  with  high  mechanical  comprehension  ask  good 
questions. 

Good  questions  converge  on  faults  and  dissect  processes. 

TRANSITIONS 

Next  project  collects  eye  tracking  data. 

Patterns  of  eye  movements  should  quickly  manifest  deep 
comprehension 


CORRELATIONS 


PREDICTOR  DEVICE  QUALITY 


COMPR. 

QA 

TA 

Spatial  reasoning 

.46 

.42 

.22 

Working  memory 

.01 

.04 

-.08 

Print  Exposure  (ART) 

.27 

.30 

-.04 

ASVAB  (g) 

.59 

.41 

.18 

Mechanical  comprehension 

.63 

.56 

.30 

Electronics 

.56 

.52 

.36 

General  science 

.60 

.48 

.24 

Auto  &  Shop 

.52 

.40 

.32 

Mathematics  knowledge 

.56 

.45 

.18 

Arithmetic  reasoning 

.52 

.37 

.11 

Numerical  operations 

.12 

.10 

.10 

Word  knowledge 

.42 

.29 

.09 

Paragraph  comprehension 

.27 

.11 

.21 

Coding  speed 

.08 

-.08 

.07 

Gender 

.37 

.37 

.30 

Age 

.01 

.01 

-.08 

Openness  (NEO) 

.32 

.24 

.08 

Neuroticism 

-.08 

.04 

-.07 

Extroversion 

-.01 

-.03 

-.03 

Agreeableness 

-.03 

-.12 

-.04 

Conscientiousness 

-.01 

-.04 

.01 

r  =  1.191  is  significant  at/7  <  .05. 
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QUESTIONS  ABOUT  A  CYLINDER  LOCK 


Mechanical  Comprehension  =  HIGH  LOW 


Is  it  the  right  key?  6  5 

What  kind  of  lock  is  it?  0  3 

Is  the  spring  broken?  5  5 

Does  spring  keep  bolt  from  moving?  3  0 

Is  the  spring  pulling  back  the  bolt?  2  0 

Is  the  spring  making  the  bolt  get  stuck?  2  0 

Is  the  cam  broken?  6  2 

Is  the  cam  moving?  2  2 

Is  the  cam  moving  back  the  bolt?  2  0 

Is  the  bar  that  fits  under  cam  broken?  4  0 

Is  the  cam  disconnected/out-of-synch  5  0 

with  the  cylinder? 

Is  the  cylinder  turning?  3  2 

Is  the  cylinder  turning  the  cam?  2  0 

Is  the  bolt  stuck  in  the  slot?  3  3 

Is  the  bolt  connected  to  the  bar?  2  0 

Are  the  pins  broken?  4  2 

Do  the  pins  lift  right?  3  2 

What  are  the  pins  used  for?  0  2 


Number  if  participants  11  11 
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